Using Replicates in Information Retrieval Evaluation
نویسندگان
چکیده
منابع مشابه
Evaluation in Information Retrieval
Over its 40 year history information retrieval evaluation evolved with an emphasis on laboratory experimentation, the “Cranfield paradigm”, in response to early demands for experimental rigour. The current and highly successful TREC experiments follow this model. However the demand for results that are robust and scalable has caused an emphasis on system-centered evaluation at the expensive of ...
متن کاملRetrieval Status Values in Information Retrieval Evaluation
Retrieval systems rank documents according to their retrieval status values (RSV) if these are monotonously increasing with the probability of relevance of documents. In this work, we investigate the links between RSVs and IR system evaluation. 1 IR evaluation and relevance Kagolovsk et al [1] realised a detailed survey of main IR works on evaluation. Relevance was always the main concept for I...
متن کاملEvaluation Issues in Information Retrieval
Evaluation techniques are critical to research in all areas of science and engineering. The major evaluation techniques used in information retrieval today were put in place over 20 years ago when Cyril Cleverdon’s Cranfield projects started an evaluation pattern that still largely dominates the field, that is, using recall/precision measures and using one of the standard test collections. Wher...
متن کاملMeasurement in information retrieval evaluation
Full-text retrieval systems employ heuristics to match documents to user queries. Retrieval correctness cannot, therefore, be formally proven, but must be evaluated through human assessment. To make evaluation automatable and repeatable, assessments of which documents are relevant to which queries are collected in advance, to form a test collection. Collection-based evaluation has been the stan...
متن کاملEvaluation Methodologies in Information Retrieval
This paper reports on the Evaluation Methodologies in Information Retrieval Seminar [1] held from 27 October to 1 November 2013 at the Schloss Dagstuhl Leibniz Center for Informatics that is a world-wide renowned venue for informatics where scientists come together to exchange their knowledge and to discuss their research findings. Dagstuhl offers modern facilities and is located in the beautif...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: ACM Transactions on Information Systems
سال: 2017
ISSN: 1046-8188,1558-2868
DOI: 10.1145/3086701